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We consider a mixture of two-species atomic Bose-Einstein codensates coupled to a bound molec- 
ular state at zero temperature via interspecies Feshbach resonance. The interspecies Feshbach 
coupling precludes the possibility of doubly mixed phases while enables not only the pure molecu- 
lar superfluid but also the pure atomic superfluids to exist as distinct phases. We show that this 
system is able to support a rich set of phase separations, including that between two distinct mixed 
atom-molecule phases. We pay particular attention to the effects of the Feschbach coupling and the 
particle collisions on the miscibility of this multi-component condensate system. 
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I. INTRODUCTION 

The experimental realization of multi-component con- 
densate systems in the late-90s [J H, H| has renewed the 
interest in the subject of miscibility of two distinguishable 
condensates in a binary mixture [1, H, H> 0> HI ■ Whether 
two condensates coexist in the same spatial volume (mis- 
cibility) or repel each other into separate spatial regions 
(immiscibility or phase separation) or even collapse when 
they are brought together is largely determined by the 
relative strength between inter- and intra-species s-wave 
scattering lengths. The ability to change the two-body 
interactions by tuning the magnetic field across the Fes- 
hbach resonance in atomic systems has enabled the ob- 
servation of phase separations, that are otherwise im- 
possible with traditional condensed matter environments, 
to be accessed experimentally. The recent experimental 
demonstration of rich phase structure including the phase 
separation in an unbalanced fermionic superfluidity sys- 
tem 0, [l(| serves as a great testament to the remarkable 
controllability that the technique of Feshbach resonance 
can bring to the atomic system. 

In this paper, we consider a model where bosonic atoms 
of two different species are coupled to a bound heteronu- 
clear molecular state via the interspecies Feshbach res- 
onance (we will simply refer to it as the heteronuclear 
model in this paper). Chiefly due to their large per- 
manent electric dipole moments, ultracold heteronuclear 
molecules are thought to hold great promise in applica- 
tions such as quantum computing and simulation 
and precision measurement of fundamental symmetry 
[H, M, [lj] and constants As such, the 

study of cold heteronuclear molecules has evolved into 
one of the most hotly pursued researches in the field of 
atomic, molecular and optical physics (l9| . The recent 
experimental efforts both in achieving the tunable inter- 
action between 41 K and 87 Rb and in demonstrating 
the ability of the Feshbach resonance to control the misci- 



bility of a 85 Rb- 87 Rb dual-species BEC j21| have further 
stimulated our interest in the proposed model. 

In an earlier paper [22| . we have concentrated on 
the coherent creation of heteronuclear molecular con- 
densates (similar studies can also be found in reference 
[HI). In this paper, we will focus on constructing zero- 
temperature phase diagrams, paying particular atten- 
tions to phase separation and collapse. In models with a 
broad Feshbach resonance, the molecular component can 
be adiabatically eliminated; this approximation leads to 
an effective binary condensate system, whose properties 
were investigated extensively in the past 0, S B S Hi • 
In general, we must treat the molecular component as an 
independent degree of freedom, regarding our model as a 
genuine three-species condensate system. However, our 
system is fundamentally different from the independent 
gas model [24|, [2|| , where no interspecies conversions can 
take place. The conversion between atoms and molecules 
can lead to phenomena unconventional to the indepen- 
dent gas model, as has been pointed out by Radzihovsky 
et al. [1|] and Romans et al. [27|. The surprise, that 
the homonuclear model admits the molecular superfluid 
(MSF) but not the pure atomic superfluid (ASF), was 
traced to the intraspecies Feshbach process which takes 
two atoms from the same species to form one molecule. 
This is no longer the case for the interspecies Feshbach 
process which requires only one atom from each species 
to produce one molecule. But, it creates a new surprise 
- the doubly mixed phases with two nonvanishing order 
parameters which are ubiquitous in the independent gas 
model are completely absent from our system. For the 
same reason, not only the pure molecular phase but also 
the pure atomic superfluids can exist in the heteronuclear 
model. This along with the possibility of simultaneously 
tuning the Feshbach energy and the atomic population 
imbalance (an extra control "knob" unique to the het- 
eronuclear model) makes the number of experimentally 
accessible phase separations (summarized in Sec. VI) far 
greater in the heteronuclear model than in the homonu- 
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clear model. 

This paper is organized as follows. In Sec. II, we de- 
scribe our model and the Hamiltonian by defining all the 
system parameters. In Sec. Ill, we derive the equilib- 
rium conditions by minimizing a total grand canonical 
Hamiltonian under the premise that the system is in an 
arbitrary state of phase separation. A general classifica- 
tion of various phases supported by our system is sum- 
marized in Sec. IV. The same section also presents the 
nonlinear coupled equation governing the mixed atom- 
molecule superfluid phase, and generalize the stability 
conditions to our three-component system. In Sec. V, 
we provide numerical examples that showcase the ability 
of our system to support a rich set of phase separation 
possibilities while at the same time elucidate the effects 
of the Feshbach coupling and the particle collisions on the 
miscibility property of our system. Finally, a summary 
is given in Sec. VI. 



II. MODEL AND HAMILTONIAN 

Our model consists of two distinct atomic species of 
state |1) and |2), and a molecular species of state |3). 
The state |3) differs in energy from the atomic states 
by an amount S. In our model, particles can interact 
with each other through collisions. At low temperature, 
we use Xij to denote the collisional strength of the s- 
wave scattering involving either same species (i = j) or 
different species (i ^ j). In addition, atoms of state |1) 
can combine with atoms of state |2) to coherently create 
molecules of state |3) via Feshbach resonance character- 
ized with atom-molecule coupling strength g and Fesh- 
bach detuning 5, tunable by changing the magnetic field. 
Let (r) be the field operator for the i-th component, 
and rtii be the mass of species i with 7713 = mi + m-i. We 
describe our system with the following Hamiltonian 
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^ [i=l,2,3 
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2rrij 



*^ (r) 



+6^1 (r) # 3 (r) + I [* 3 (r) *i (r) * 2 (r) + h.c 



E 



^*+(r)*t( r )£ j(r) * 4 (r) 



(1) 



which can be easily generalized from the Hamiltonian for 
the homonuclear model 0, [2tJ ■ 

In this paper, we limit our study to the homoge- 
neous condensate system at zero-temperature where each 
species has already condensed into the zero momentum 
mode. As such, we can describe such a system in the 
spirit of the mean-field approximation in which ^ (r) is 
replaced with the uniform c- number ipi exp (itpi), where 
tpi > and ipi are two real numbers, representing, respec- 
tively, the modulus and the phase of the corresponding 
complex order parameter. It can be easily shown that 



the mean-field energy density £ =(H)/V (with V being 
the total volume) is a function of the "relative phase" 
ip = tpi + (f2 — y>3 and is minimized when <p = it (as- 
suming, without loss of generality, that g > 0). With 
these considerations, we derive from Eq. |T]) the follow- 
ing mean-field (low branch) energy density 



£ 



e ¥m> 



(2) 



which will be used in the next section as the starting 
point for studying phase and phase separations. 



III. GENERAL CONDITIONS FOR PHASE 
SEPARATION 

To begin with, we imagine that our system at the 
ground state is phase separated into a number of arbi- 
trary phases (yet to be determined), each of which oc- 
cupying a volume of Vi. For the purpose of easy illus- 
tration, without loss of generality, we consider the case 
that the system separates into two different phases. The 
total energy of the system, when the interface energies 
are ignored Q , then becomes 



E, 



total 



V x £^ + V 2 £^\ 



(3) 



where £^ is the mean- field energy density of the nth 
phase and is equivalent to £ in Eq. Q when ipi is re- 
placed with ij) 1 ^ , the order parameter of the ith compo- 
nent in the nth phase. The ground state energy without 
phase separation is then a special case where, for exam- 
ple, V 2 = and Vx = V. 

To determine the ground state, we construct a gener- 
alized (mean-field) grand canonical Hamiltonian 



Ktotai = Etatal - l*N - hN d - mV 



(4) 



where fi, h, and m are three Lagrangian multipliers as- 
sociated, respectively, with the conservation of the total 
particle number 



Ef(^ B) ) a + K ) ) a+2 (^ B) ) : 

n=i L 

the total atom number difference 
2 



V n = N. (5) 



71=1 



E (# } ) -W 



V n =N d 



and finally the total volume 

Vi + Vi = v. 



(6) 



(7) 



To proceed further, we first define the free energy den- 
sity 

K = - J2 ^ - &Mwh + E ■ ( 8 ) 
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When the use of Eqs. Q, (O, and ([7]) is made, we find 
that 

Ktotai = ^i/C (1) + V 2 JC {2) - m (Vi + V 2 ) , (9) 

where K,^ is equivalent to JC in Eq. (|8|) with ipi and fii 
substituted with ip^ and fj,\ , respectively. In Eq. (jSJ), 

(1) (2) 

Mi — Mi = M^ 

with /^i = (j, + h, fj,% = /j, — h, and fj,s = 2fj, — 5. Thus, 
as expected, the chemical potentials in equilibrium are 
balanced as a consequence of the particle number conser- 
vation laws [Eqs. §5§ and ©]. Minimization of K to tai 
with respect to Vi and V2 results in 

K^=K^. (10) 

Thus, we conclude that (1) for such a phase separa- 
tion to be possible, phase 1 and phase 2 must coexist 
for a given set of /Xi, \i 2 and fi^ or must overlap in the 
chemical potential space, and (2) phase separation takes 
place along the first-order phase transition line where the 
energy density of phase 1 equals that of phase 2 [28j . Ev- 
idently, the same conclusion holds for phase separation 
involving an arbitrary number of phases. 

IV. POSSIBLE PHASES AND STABILITY 
CRITERIA 

A. Possible Phases 

In order to determine the possible phases, we minimize 
Eq. ^ with respect to ip^ . This procedure results in 
the following set of saddle-point equations, 

i 
i 

M3^3 = --V'l^ + *p3 ^2x3ini- (11) 

i 

where we have replaced ip^ with ipi and used rii = ipf to 
denote the density of the ith species. The possible phases, 
which correspond to different solutions of Eqs. ijTTjl . can 
then be broadly divided into five groups: 

(i) Vacuum with ipi = ip2 = ip3 = 0; 

(ii) Atomic superfluid of species 1 (ASF1) with ipi — 

y/m/xn, i>2 = i>3 = 0; 

(hi) Atomic superfluid of species 2 (ASF2) with ip 2 = 

A/M2/X22, ipi=i>3 = 0; 

(iv) Molecular superfluid (MSF) with ^3 = \/Jl3~Jx33, 
ipi = i>2 = 0; 

(v) Mixed atom-molecule superfluid (AMSF) with 
i>\ ^0,ip 2 ^0 and ^3 ^ 0. 



We first note that our model allows the existence 
of both MSF and ASF, in contrast to the homonu- 
clear model, which supports MSF but not ASF. In our 
model, the energy density due to the Feshbach coupling 
is g?piip2ip3i which is symmetric with respect to the ex- 
change of any pair of states and hence does not favor the 
formation of one pure phase to another. In the homonu- 
clear model, on the other hand, the energy density due 
to the Feshbach coupling is in the asymmetric form of 
gip3ipf. With such a form, condensation of atoms will 
inevitably lead to the formation of the molecular con- 
densate while condensation of molecules can take place 
in the absence of the atomic population. 

We emphasize that this classification holds only in the 
presence of the Feshbach coupling (g 7^ 0). In a typi- 
cal three interacting gas system where g = 0, the sys- 
tem supports, besides all the phases listed above, doubly 
mixed phases, for example, ASF1-MSF in which tpi 7^ 0, 
ip3 7^ 0, and ^2 = 0. The absence of the doubly mixed 
phases can again be attributed to the unique form of the 
Feshbach coupling, (7'0iV'2'03j m the heteronuclear model. 
This term dictates that the existence of condensate pop- 
ulation in any pair of states will lead to the condensation 
in the third component, which precludes the possibility 
of forming the doubly mixed phases where the population 
in the third component is zero. 

We also comment that for a given set of chemical po- 
tentials, while ASF1, ASF2, and MSF are unique, AMSF, 
due to the highly nonlinear nature of Eq. (|lip . may itself 
be divided into different types depending on the density 
distribution among three species. Such an example will 
be given in Sec. EH 

Finally, this classification, although obtained under the 
condition of zero temperature, can provide a useful road 
map for understanding the phase diagrams at finite tem- 
perature. At finite temperature, state (i) becomes the 
so-called normal state where the gas mixture does not 
possess any order, and other states become mixed in the 
sense that superfluidities and thermal densities are both 
present. Lowering temperature reduces the entropy and 
restores the order to the gas system, thereby giving rise 
to a hierarchy of critical temperatures associated with 
phase transitions that break symmetries at various lev- 
els. The subject of finite temperature phase diagrams 
will be left as a future study. 

B. Stability Criteria 

From Eq. (|11[) we have determined the possible phases, 
which correspond to the critical points (extrema or sad- 
dle points) of the free energy density /C. We now turn our 
attention to studying their stability properties by iden- 
tifying the local minimum of the function IC. For this 
purpose, we first introduce the Hessian matrix 

(fll fl2 fl3 \ 
fi2 hi f23 , (12) 
/l3 /23 /33 / 
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where the matrix element is defined as = d^d^K, 
which, with the help of Eq. ([5]), are shown to have the 
form 

/l2 = -gi>3 + 4X12^1^2, 

/l3 = -gip2 + 4X13^1^3, 

/23 = -91pl + 4X23^2^3, 
3 

fa = ^Xu n i + 2 ^Xijrij - 2m. 

2=1 

Note that we have defined the partial derivatives in the 
Hessian matrix to be taken with respect to ipi instead of 
m so that we can carry out the stability analysis of both 
pure and mixed phases in a unified manner. 

A state (phase) is thermodynamically stable if it is 
a local minimum of the function K. for a given set of 
chemical potentials. This requires the Hessian matrix to 
be positive semidefinite, which, for a symmetric matrix 
as in Eq. (fT2")) , is possible if and only if all the principal 
minors of Hessian matrix are nonnegative. This leads to 
a hierarchy of stability conditions 

fa > 0, (13a) 
/«/#-/£•> 0, (i^j), (13b) 
/11/22/33 + 2/12/13/23 

-/ll/ 2 2 3 " /22/13 " /33/l 2 2 > 0, (13c) 

which reflects the underling philosophy that for a three- 
component miscible system to be stable, all the subsets 
with single, double, and triple components must also in- 
dependently satisfy the relevant stability conditions [24| • 

V. RESULTS AND DISCUSSIONS 

In this section, we construct phase diagrams and dis- 
cuss phase separations under a variety of conditions. 
In all the numerical examples, we use two quanti- 
ties, no with the dimension of the density and £q = 
h 2 n^/ 3 (mi + 777*2) /t77i 7772 with the dimension of the en- 
ergy, to define a scaling which consists of the following 
set of dimensionless variables: K = JC/SqUq, fii = rij/no, 
= 4>i/\/no-, pi = /ii/£b,5 = 5/£ a , g = g^/no/So, and 
Xij = Xij n o/£o- To gain a familiarity with the param- 
eters, we consider the interspecies Feshbach resonance 
around 35 G in the 41 K- s7 Rb mixture, which is char- 
acterized with a magnetic field width AB = 5.1 G, a 
magnetic moment difference A/_t = 0.005«_b (fJ-B being 
the Bohr magneton) , and a background scattering length 
ab g = 284a (00 being the Bohr radius) [2(1 H§| . For a 
typical density no = 10 15 cm -3 , the energy unit is around 
£ w 2.4 x 1(T 29 J. Thus, g = v / 27rh 2 a bg ABA^/m 12 , 
when converted into the unitless form, becomes g w 0.96. 
Additionally, based on an = 99ao, a.22 = 60ao, and 012 = 
640a 29] where we have designated 87 Rb as species 1 and 
41 K as species 2, we estimate that xn ~ 0.2, X22 ~ 0.3, 
and X12 ~ 2.13. 



In our discussions below, we first map out the phase 
diagrams in the chemical potential space. This includes 
solving Eqs. (jTTJ) to obtain different phases and to per- 
form stability analysis according to the criteria (|13[) . 
These diagrams enable us to identify the regions where 
different phases overlap and hence help to answer the 
question of into which phases an unstable homogeneous 
system will separate. However, from the experimental 
point of view, more useful are the phase diagrams in the 
S-d (= n% — n 2 ) space, which are directly accessible to ex- 
periments via two control "knobs", the Feshbach detun- 
ing 5 and the two species atomic condensate population 
difference d. 



A. Collisionless Limit 

In order to identify the roles of collisions more clearly 
in later sections, we first consider a situations where we 
artificially turn off all the s-wavc scatterings. For peda- 
gogical reasons, here we only present the phase diagrams 
in the S-d space, which are in practice translated from 
the phase diagrams in the chemical potential space (see 
the example in Sec. IV Bl for more details regarding the 
mapping between the two types of diagrams.) 



1. Without Feshbach Coupling (g = 0) 

In the collisionless limit and without the Feshbach cou- 
pling (g — 0), our system simplifies to a three-species 
ideal gas model, where Eqs. (jTTJ) reduces to 

mi>i = 0. (14) 

Figure[JJa) is the corresponding phase diagram in the S-d 
space. This model is simple enough that the features in 
Fig. [TJa) are almost self-explanatory. Take as an exam- 
ple the doubly mixed phase ASF1-MSF with %j) X ^ 0, 
t/>3 7^ 0, and ip2 = 0. Such a phase is possible only 
when ii\ — ^3 = but 112 is arbitrary according to Eqs. 
(fT4)) . This means that d = ip\ — ip% = ip\ > and 
S = [ii + /J2 — A*3 = M2 < 0, where the inequality is due to 
the stability criterion (|13a|) . This is why the top left re- 
gion of Fig. [TJa) hosts ASF1-MSF. Other features can be 
similarly explained. We note that Fig. [TJa) has similar 
properties as in Fig. 5 of Ref. [25|. The physics is very 
clear: as the Feshbach detuning is tuned from positive to 
negative or equivalently as the molecular energy level is 
lowered from above to below the atomic levels, the sys- 
tem makes a transition from the atomic side with atomic 
phases to the molecular side dominated with phases in- 
volving the molecular component. 
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FIG. 1: The phase diagrams in the 5-d space where (a) and (b) 
are produced with g — for (a) and g = 1 for (b), and (c) and (d) 
correspond respectively to (a) and (b) while collisional coefficients 
are set at Xii = 0.2, %22 = 0.3, X12 = 2, and j(3i = 0. See 
the text for units. For the sake of clarity, the labels for ASF1 and 
ASF2, which are located at d = 1 and d = —1, respectively, are 
omitted. 



2. With finite Feshbach Coupling (g ^ 0) 

Figure QJb) is the phase diagram in the 5-d space, 
when the Feshbach coupling is turned on. Noticeable 
changes, compared to Fig. [2a) where g — 0, are the dis- 
appearance of the doubly mixed phases into the AMSF 
phase, and the corresponding transition boundaries into 
an atom-molecule crossover region (see Sec. IIV Al for the 
explanation). In addition, the MSF transition is shifted 
to 5 = owing to the modification of the molecu- 

lar binding energy by the Feshbach coupling (2|| H3, H3] • 

We explain below that without collisions, the AMSF 
phase will collapse. Let us begin with Eqs. (fTTj) which, 
in the absence of collisions, reduce to 2/ziV>i = — (/V^V^i 
2/12^2 = — ffV'iV'Si 2^3-03 = —gipii^2- Thus, within the 
chemical potential space defined by 



Hi < 0, [i 2 < 0, H3 < (or /ii + /i 2 < 5), 



(15) 



there exists a physical AMSF solution (ipi > 0) with a 
density distribution 



1 ' 

g 



n 2 



9 '■ 

9 



n 3 



9 2 ' 



(16) 



This AMSF phase is unstable because the left-hand side 
of the stability condition (|13cp is found to be 32/^1/^2/^3, 
which turns out to be less than zero as a result of the 
condition (fT5jl . The only other homogeneous solution 
besides this AMSF is the vacuum, which is stable within 
the space defined by Eq. (fT5")) according to Eq. (|13a[) . 
Thus, we conclude that the only route for this unstable 
AMSF to take is to collapse. The system may, however, 
be stabilized by kinetic energy without going into a com- 
plete collapse, exhibiting the so-called rarified liquid-like 



behavior [3l|. The dynamics of phase separation and 
collapse may lead to quite complex spatial patterns [32| . 



B. Collisions Involving Atoms Only 

In general, collisions greatly complicate the construc- 
tion and understanding of the phase diagrams, because 
Eqs. (jlip then become highly nonlinear and may support 
several AMSFs. In light of the fact that the collisional 
coefficients involving the molecular state are not known 
in most experiments, we ignore in this subsection all the 
collisions involving the molecular component {\a = 0), 
and take into consideration only the atomic collisions. 
In particular, we focus our discussion on the case with 
Xn = 0.2 and X22 = 0.3, and X12 = 2, where the in- 
terspecies collision is far stronger than the intraspecies 
collisions. 



1. Without Feshbach Coupling (g = 0) 

Figure [TJc) is the phase diagram in the 5-d space when 
g = 0. Figure [TJc) contains, in addition to all the phases 
occurred in Fig. [TJa) where collisions are turned off, a 
unique AMSF given by 
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(17) 



n 3 = (1 — ni - n 2 ) /2, and /i 3 
rounded by three lines, 



d 
d 
d 



0. This AMSF is sur- 



X12 + X22 

5 



X12 +X11 

2(5 - xii - 2xi2 



X22 



Xii - X22 



serving as the borders between AMSF and MSF, ASF1- 
MSF, and ASF1-ASF2, respectively. These equations are 
obtained from the conditions that 7ij = (i = 1, 2, 3), re- 
spectively. Evidently, all the lines are collapsed into a 
vertical line at 6 = and no AMSF is possible in the 
absence of collisions as in Fig. [lja). Understandably, 
ASF1-ASF2 and its next level of hierarchy, AMSF, are 
both expected to be unstable against phase separation 
due to the strong repulsion between the two atomic com- 
ponents (1,|2|| . 



2. With finite Feshbach Coupling (g 7^ 0) 

In the case of finite Feshbach coupling, we first obtain 
from Eqs. (fTTJ) that 



03 



9 
2/i 3 



^1^2, 



(18) 
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which, when substituted into the other two equations, 
gives a unique AMSF solution with the following atom 
density distribution 



X11X22 



v '2 
A12 



n 2 



X11M2 - X12M1 



X11X22 



(19) 



where x'12 — X12 + 5 2 /4^3 plays the role of an effective 
interaction between the two atomic species. The extra 
term g 2 /4/Z3 can be traced to the Feshbach coupling; an 
atom of species 1 appears to interact with an atom of 
species 2 via the molecular state by the Feshbach coupling 
of the strength g. This interaction becomes virtual when 
8 is large but can become substantial when 5 is near 
resonance. Unlike the AMSF in the case of g = 0, where 
[13 = 0, 113 here must be less than zero (/U3 < 0). Thus, 
we always have x'12 < Xi2- It is well known that a 
double BEC mixture is stable when x'12 < X11X22, and 
otherwise, it is unstable and will separate when x'12 > 0> 
and collapse when x'12 < [S Hi]- 

Figure [TJd) shows the phase diagram in the 5-d space 
when g is finite. For the reason given previously, we ob- 
serve that the regions of doubly mixed phases give away 
to the AMSF crossover region. With our choice of Xiji 
the system has a natural tendency to phase separate. 
However, we see that a narrow strip of stable AMSF 
emerges from the region near the Feshbach resonance. 
This is possible because x'12 can be substantially smaller 
than X12 while remaining positive. This illustrates how 
one can change the mixture from immiscible to miscible 
by varying the Feshbach detuning [2lj . 

In addition, in the region around the MSF line, be- 
cause most populations will accumulate in the molecular 
component, ^3 = —gipi^/^s will take a small nega- 
tive value, which in turn makes the contribution from 
g 2 /4/i3 to the effective interparticle interaction x'12 rela- 
tively strong. So the system will collapse in this region 
but phase separate outside this region where the repul- 
sive interaction become dominant again. 
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FIG. 2: (Color online) The phase diagrams in the 5 — d space (the 
upmost figure, with total density fit = 1), the chemical potential 
space [(a) and (b)], and the density space [(c) and (d)] when g = 1, 
Xn = 0.2, X22 = 0.3, xaa = 0.245, X12 = 0.1225, and X n = 
X23 = 0. Additionally, (a) and (c) are produced with S = 0.5 
while (b) and (d) are produced with 5 = —0.8. The gray area 
in (a) and (b) represents the region where two stable phases can 
coexist. The red dashed lines inside the gray area are the first-order 
phase transition lines at which the ground state changes from one 
phase to the other. See the inset for more details of the region 
around the first-order line. Consult the text for units and for a 
detailed explanation of each diagram. 



C. Collisions Involving Both Atoms and Molecules 

The main new feature when collisions involving 
molecules are also included is that more than one AMSFs 
may emerge. To demonstrate this feature, we consider, 
for simplicity, a model in which a finite X33 is introduced 
in addition to the atomic collisional terms (while still 
keeping Xi3 = X23 = 0). By solving Eqs. (jlip. we find 
that the AMSF phase is governed by equations same as 
Eqs. (jT8|) and (fl9|) except that /X3 is now replaced with 
fi' 3 = /j,3 — X33 n 3- As can be seen, the chemical poten- 
tial is now modulated by a Kerr nonlinear term —X33 n 3- 
It is well known that the Kerr nonlinear term can lead 
to multiple solutions that can result in multi-stability in 
nonlinear systems. Here, we find (not shown) that 713 is 
governed by a fifth-order polynomial equation, and hence 
may support more than one physical solutions. 



1. One Stable AMSF and One Unstable AMSF 

Let us consider here a case where we choose the 
molecule-molecule collisional coefficient to be %33 = 
0.245, and all the atomic collisional parameters same as 
in Figs, [ijc) and (d) except that the interspecies repul- 
sion is now set at a moderate value of X12 = 0.1225. The 
phase diagram in the 5-d space for this case is displayed 
in the upmost figure of Fig. [21 which is all occupied by 
the AMSF state except for the line at d = marked with 
MSF. Because of the much reduced X12, the system has a 
natural tendency to be in a miscible state and the AMSF 
will not phase separate (the white area) except in a small 
parameter region (the grey area) close to the Feshbach 
resonance. Unlike in Fig. [T] where we continue to label 
the phase separated region as AMSF, in Fig. ^ we will 
not show the unstable AMSF, instead, we divide it into 
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regions and label them according to how the homoge- 
neous (but unstable) AMSF is going to phase separate. 
For example, AMSF-ASF1 means that the homogeneous 
AMSF within the grey region of that label will phase 
separate into one AMSF and one ASF1. 

The origin of this rich set of phase separations can be 
best understood from the chemical potential and density 
space. Consider, for example, 8 = 0.5. In Fig. HJa), 
we construct the phase diagram in the chemical poten- 
tial space. For a fixed set of chemical potentials, the 
coupled equations (jlip are then found to support two 
physical AMSFs in some parameter regimes: one is sta- 
ble and the other is unstable. Only the stable AMSF is 
shown here. As Fig. [Ha) illustrates, the stable AMSF 
coexists and shares the first-order transition line, respec- 
tively, with ASF1, ASF2, and vacuum. The first-order 
transition line in Fig. [Ha) is transformed into the grey 
region in Fig. [2tc) when the phase diagram is mapped 
from the chemical potential to the density space [33l . [3J| . 
For our purpose, we find it convenient to define the den- 
sities nit = fix -\- r%3 and nn = n% + n.3 so that nit + nn 
and nu — nit (= d) represent the total density and the 
population imbalance, respectively. For a system oper- 
ating within the grey region in Fig. Etc), it will phase 
separate into AMSF and ASF1, or AMSF and ASF2, and 
or AMSF and vacuum. The separated phases in each set 
have different densities, and phase separation is a natural 
course for a system to take in order to maintain the av- 
erage densities specified within the respective domain of 
the phase separation. The phase separation into AMSF- 
vacuum signals collapse. 

As 8 is reduced, for example, to 8 = —0.8, a new phase 
separation, AMSF-MSF, appears in the upmost figure, 
which is again supported by Figs. Etb) and (d) in the 
chemical potential and density space. Compared to 8 = 
0.5 where the AMSF is unstable irrespective of the value 
of d, the AMSF at 8 = —0.8 becomes a stable one when 
the population imbalance between the two atomic species 
is sufficiently large. 



2. Two Stable AMSFs and One Unstable AMSF 

The nonlinearity of Eqs. may provide the oppor- 
tunity for an unstable homogeneous AMSF to phase sep- 
arate into two different AMSFs. Consider the model that 
shares the same parameters as in Fig. [2]except that \i2 is 
set back to a value (xi2 = 2) same as in Fig. Q] It is found 
that Eqs. (|TT|) support three physical AMSF solutions, 
two of which satisfy the stability conditions (flU)) and are 
hence stable. The phase diagrams corresponding to this 
model are shown in Fig. [3] A coexistence region between 
two stable AMSFs having different population distribu- 
tions is identified from the chemical potential space both 
for 5 = 1 [Fig. Ha)] and 8 = 0.5 [Fig. Efb)]. However, in 
experiments where the total population is fixed, AMSF1- 
AMSF2 is accessible only to sufficiently small Feshbach 
detunings, for example, 8 — 0.5 (see the phases crossed 







AMSF-ASF2— y 
AMSF-AMSF-/. 


ksF1-ASF? 


1 

MSF \ 

AMSF-ASF1- 



-2-10 1 2 



_ s _ 

S =\ 5=0.5 




FIG. 3: (Color online) The phase diagrams similarly defined as in 
Fig. 2. The parameters are the same as those in Fig. 2 except 
that "jjjxa — 2. Additionally, (a) and (c) are produced with 5 = 1 
while (b) and (d) are produced with 5 = 0.5. The red dashed 
lines have the same meanings as that in Fig. 2. Consult the text 
for units. 

by the (imagined) line fin + nit = 1 in both Fig. E{c) 
and Fig. |3fd)]. We emphasize that so far, we have not 
seen reports of phase separation into two AMSFs both 
in theory and in experiments. In particular, we mention 
that the homonuclear systems are known to only sup- 
port phase separations involving one AMSF {26]. For 
this reason, we suspect that this may be a phenomenon 
quite unique to the heteronuclear model. 

This case also provides an example for us to see the ef- 
fect of the molecule-molecule interaction on the stability 
of the AMSF. Compared to Fig. [LJd) where x 33 = 0, 
the upmost diagram of Fig. [3] shows that the presence 
of X33 stabilized the AMSF on the molecular side of the 
Feshbach resonance. 



VI. SUMMARY 

In conclusion, we have studied the zero-temperature 
mean-field phase diagrams of two-species atomic BEC 
mixtures with the interspecies Feshbach resonance un- 
der a variety of conditions, with a special attention be- 
ing given to the subject of phase separation. We have 
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found that on one hand, the Feshbach coupling alone 
cannot create a stable AMSF; to create stable AMSF, 
collisions must be present, and on the other hand, the 
Feshbach coupling may stabilize a system that has a nat- 
ural tendency to phase separate under collisions. We 
have shown that the molecule-molecule collision can sta- 
bilize the AMSF on the molecular side of the Feshbach 
resonance. We have identified the interspecies Feshbach 
interaction to be the cause for the absence of the doubly 
mixed phases, as well as the reason for the presence of 
both pure atom and pure molecule phases in our model. 
This provides the opportunity, as we have indeed veri- 
fied numerically, for observing the phase separation not 
only between AMSF and MSF but also those between 
AMSF and pure ASFs. We have also studied the col- 
lapse or partial collapse of the AMSF as special phase 
separations where one of the separated components is the 
vacuum. Under certain conditions, we have even found 
that our system is able to phase separate into two dis- 
tinct AMSFs, which, we speculate, is a property unique 
to the heteronuclear model. 



No. 10474055, the National Basic Research Program of 
China (973 Program) under Grant No. 2006CB921104, 
the Science and Technology Commission of Shanghai 
Municipality under Grant No. 06JC14026 and No. 
05PJ14038, the Program of Shanghai Subject Chief 
Scientist under Grant No. 08XD14017, the Program 
for Changjiang Scholars and Innovative Research Team 
in University, Shanghai Leading Academic Discipline 
Project under Grant No. B480, the Research Fund for 
the Doctoral Program of Higher Education under Grant 
No. 20040003101 (W.Z.), by China Postdoctoral Science 
Foundation under Grant No. 44021570 (L.Z.) and by 
the US National Science Foundation (H.P., H.Y.L.), the 
US Army Research Office (H.Y.L.), and the Robert A. 
Welch Foundation (Grant No. C-1669), and the W. M. 
Keck Foundation (H.P.). 



t wpzhang@phy.ecnu.edu.cn 



Acknowledgments 



* ling@rowan.edu 



This work is supported by the National Natural Sci- 
ence Foundation of China under Grant No. 10588402 and **hpu@rice.edu 



C. J. Myatt, E. A. Burt, R. W. Christ, E. A. Cornell, 
and C. E. Wieman, Phys. Rev. Lett. 78, 586 (1997). [16 

D. S. Hall, M. R. Matthews, J. R. Ensher, C. E. Wieman, 
and E. A. Cornell, Phys. Rev. Lett. 81, 1539 (1998). [17 
J. Stenger, S. Inouye, D. M. Stamper-Kurn, H.-J. Mies- 
ner, A. P. Chikkatur, and W. Ketterle, Nature (London) [18 
396, 345 (1998). 
T. L. Ho and V. B. Shenoy, Phys. Rev. Lett. 77, 3276 [19 
(1996). 

P. Ao and S. T. Chui, Phys. Rev. A 58, 4836 (1998). 

B. D. Esry, C. H. Greene, J. P. Burke, Jr., and J. L. [20 

Bohn, Phys. Rev. Lett. 78, 3594 (1997). 

H. Pu and N. P. Bigelow, Phys. Rev. Lett. 80, 1130 

(1998); 80, 1134 (1998). [21 

E. Timmermans, Phys. Rev. Lett. 81, 5718 (1998). 

M. W. Zwierlein, A. Schirotzek, C. H. Schunck, and W. [22 
Ketterle, Science 311, 492 (2006). 
G. B. Partridge, W. Li, R. I. Kamar, Y. A. Liao, and [23 
R. G. Hulet, Science 311, 503 (2006); G. B. Partridge, 
W. Li, Y. A. Liao, R. G. Hulet, M. Haque, and H. T. C. [24 
Stoof, Phys. Rev. Lett. 97, 190407 (2006). 

D. DeMille, Phys. Rev. Lett. 88, 067901 (2002). [25 
A. Micheli, G. K. Brennen, and P. Zoller, Nature Phys. 
2, 341 (2006). [26 
M. G. Kozlov, and L. N. Labzowsky, J. Phys. B 28, 1933 
(1995). [27 
J. J. Hudson, B. E. Sauer, M. R. Tarbutt, and E. A. 
Hinds, Phys. Rev. Lett. 89, 023003 (2002). [28 
D. DeMille, S. Sainis, J. Sage, T. Bergeman, S. Ko- 
tochigova, and E. Tiesinga, Phys. Rev. Lett. 100, 043202 [29 



(2008). 

V. V. Flambaum, M. G. Kozlov, Phys. Rev. Lett. 99, 
150801 (2007). 

T. Zelevinsky, S. Kotochigova, and J. Ye, Phys. Rev. 
Lett. 100, 043201 (2008). 

D. DeMille, S. B. Cahn, D. Murphree, D. A. Rahmlow, 
and M. G. Kozlov, Phys. Rev. Lett. 100, 023003 (2008) 
J. Doyle, B. Friedrich, R. V. Krems and F. Masnou- 
Seeuws, Eur. Phys. J. D 31, 149 (2004); J. M. Hutson 
and P. Soldan, Int. Rev. Phys. Chem. 25, 497 (2006). 
G. Thalhammer, G. Barontini, L. De Sarlo, J. Catani, F. 
Minardi, and M. Inguscio, Phys. Rev. Lett. 100, 210402 
(2008). 

S. B. Papp, J. M. Pino, and C. E. Wieman, Phys. Rev. 
Lett. 101, 040402 (2008). 

L. Zhou, W. Zhang, H. Y. Ling, L. Jiang, and H. Pu, 
Phys. Rev. A 75, 043603 (2007). 

M. Duncan, A. Foerster, J. Links, E. Mattei, N. Oelkers, 

and A. P. Tonel, Nucl. Phys. B 767, 227 (2007). 

D. C. Roberts and M. Ueda, Phys. Rev. A 73, 053611 

(2006). 

T. Nishimura, A. Matsumoto, and H. Yabu, Phys. Rev. 
A 77, 063612 (2008). 

L. Radzihovsky, J. Park, and P. B. Weichman, Phys. Rev. 
Lett. 92, 160402 (2004). 

M. W. J. Romans, R. A. Duine, S. Sachdev, and H. T. 
C. Stoof, Phys. Rev. Lett. 93, 020405 (2004). 
L. Radzihovsky, P. B. Weichman, and J. I. Park, Annals 
of Physics 323, 2376 (2008). 

A. Simoni, M. Zaccanti, C. D'Errico, M. Fattori, G. 



9 



Roati, M. Inguscio, and G. Modugno, Phys. Rev. A 77, 
052705 (2008). 

[30] R. A. Duine and H. T. C. Stoof, Physics Reports 396, 
115 (2004). 

[31] E. Timmermans, P. Tommasini, R. Cote, M. Hussein, 
and A. Kerman, Phys. Rev. Lett. 83, 2691 (1999); E. 
Timmermans, P. Tommasini, M. Hussein, and A. Ker- 
man, Phys. Rep. 315, 199 (1999). 



[32] S. Ronen, J. L . Bohn, L. E. Halmo, M. Edwards, 

larXiv:0807.4980l 
[33] D. E. Sheehy and L. Radzihovsky, Annals of Physics 322, 

1790 (2007). 

[34] F. M. Marchetti, C. Mathy, M. M. Parish, and D. A. 
Huse, Phys. Rev. B 78, 134517 (2008). 



